Robust statistical methods and software for meta-analysis with outlying studies Principal Investigator: Haitao Chu, M.D., Ph.D. Summary The mission of the Agency for Healthcare Research and Quality (AHRQ) is to produce unbiased and precise evidence to make health care safer, more accessible and affordable. The rapid growth of interest in comparative effectiveness research (CER) and evidence-based medicine (EBM) has led to a dramatic increase in attention paid to systematic reviews and meta-analyses, which synthesize and contrast multiple studies into a form of evidence that can be used to underpin guidelines, patient decision aids, and other products that shape health care. Outlying studies frequently appear in systematic reviews and meta-analyses even with stringent inclusion and exclusion criteria. However, traditional meta-analysis approaches cannot effectively and efficiently handle outlying studies. Specifically, conventional approaches commonly include two potentially iterative steps: 1) to detect and identify outlying studies; 2) to conduct sensitivity analyses by excluding the outlying studies. This two- step approach may need to be examined iteratively as the decision rule to determine whether a study is an outlier may change after we exclude ?outlying? studies. Furthermore, different outlier detection methods may identify different studies as ?outliers?. Thus, this approach may not be efficient given the ever increasing volume of studies and increasing demand for timely synthesized evidence, and may lead to inconsistent results when the results differ significantly when outlying studies are included versus excluded, or when different outlier detection methods identify different outliers. In response to PA-15-147, the overall goal of this proposal is to develop robust statistical methods to better estimate overall effect size in meta-analyses with a few outlying studies. In the presence of outlying studies, the between-study variance is commonly overestimated. In consequence, the commonly used methods to estimate the overall treatment effect can be sensitive to outlying studies, which in turn will impact the validity and reliability of systematic reviews and meta-analyses. In this proposal, we will focus on developing innovative statistical methods and software to accurately estimate overall mean effect size using penalized-likelihood based methods, which take the advantages of both the conventional fixed-effect and random-effects models. In addition, we propose to empirically examine the performance of the proposed and existing methods by reanalyzing a randomly selected 2,300 meta-analyses published in the Cochrane Database of Systematic Reviews. Furthermore, the performance between the proposed and existing methods will be rigorously investigated using carefully designed extensive simulations. User-friendly and freely available software will be developed for users to implement our proposed methods accurately and easily. The completion of this proposal will substantially advance CER and EBM through innovative meta-analysis methods. It will improve public health by facilitating treatment selection for various cancers, and for cardiovascular, infectious, and other diseases.